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1. Apparat:us for archiving cLat:a defining models of 
tihree^dimensional objectis^ comprising s 
5 a dat:a r-eceiver for receiving dalia defining 

movements of -the three-dimensional objects; 

a first archive data generator for generating first 
archive data defining where at least one object is 
looking; and 

10 a dataJi>ase for storing the first archive data such 

that it is associated with archive data for the three- 
diinens±onal objects - 



2. Apparatus according to claim 1, wherein the first 
15 archive data generator is arranged to generate first 

archive data defining a person, animal or object at which 
the at least one object is looking « 

3. Apparatus according to claljn 1^ wherein t-he database 
20 is arranged to store archive data for the three- 
dimensional objects comprising data defining the 
movements thereof . 
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4. Apparatus according to claim 1, wherein the database 
is arranged to store archive data for the three- 
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dimensional objects comprising data defining the three- 
dimensional objects in different positions. 

5. Apparatus according to claim 1, wherein the first 
archive data generator is arranged to generate respective 
first archive data for data received at different times 
by the receiving means and the database is arranged to 
store the first archive data associated with the 
corresponding archive data for the t.hree-dimensional 
objects and associated with second archive data 
comprising timing data. 

6. Apparatus according to claim 1, further comprising 
^ third archive data generator for generating third 
archive data defining text corresponding to words spoken 
by one of the tliree-dimensional objects, and wherein the 
database is arranged to store the third archive data such 
that it is associated with the archive data for the 
three-dimensional objects and related archive data. 

7. Apparatus according to claim 6^ wherein the third 
archive data generator comprises an audio receiver for 
receiving audio data and an audio data processor for 
processing the audio data to generate the text data 
therefrom. 
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8* Apparatus accor<aing i:o claim 1, wherein tihe data 
receiver means is further arranged to receive audio data, 
and wherein the database Is arranged to store the audio 
data such that It Is associated with the archive data for 
5 the three-dimensional objects and related archive data. 

9. Apparatus according to claim 1, wherein the database 
is arranged to store the archive data for the three- 
dimensional objects as MPEG 4 data* 

10 

10. Apparatus according to claim 8r wherein the database 
is arranged to store the archive data for the three- 
dimensional objects and the audio data as KFBG 4 data* 

15 11. Apparatus according to claim 1, further comprising 

a fourth archive data generator for generating fourth 
aorchive data defining, for a predetermined period, the 
proportion of time spend by a given three-dimensional 
object looking at each of other three-dimensional objects 

20 during the predetermined period, and wherein the database 

is arranged to store the fourth archive data such that it 
is associated with the archive data for the three- 
dimensional objects and related archive data* 
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12. Apparatus according to claim 11, wherein the 
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prede-termined period comprises a period during which the 
given tihree-dijnensional object was tallciag- 

13* Apparatus according to claim 1, further comprising 
a movement data generator for processing Image data from 
at least one camera to generate data defining movements 
of the three-dimensional objects. 

14. Apparatus according to claim X, further comprising: 
a store for storing data defining three-dimensional 

models of the ob^fects; and 

a model data processor for changing the data In the 
store to give amended model data in dependence upon the 
data received by the data receiver defining movements of 
the objects; 

and wherein the database is arranged to store the 
amended model data as the archive data for the three* 
dimensional objects. 

15. Appairatus according to claim 1, further comprlsdLng 
a database searcher for searching data stored in the 
database In accordance with search instructions to 
identify each part of the archive data for the three- 
dimensional objects which meets criteria defined in the 
search instructions. 



183 2635650 (CFP 1194 US) 

16. Appara-tus according to claim 15, wherein the 
database searcher is operable to search data stored in 
the dateibase in accordance with search criteria relating 
to the first archive data. 

5 

17. Apparatus according to claim 5, further comprising 
a dataibase searcher for searching data stored in the 
database in accordance with search instructions to 
identify each part: of the archive data for the -three- 

10 dimensional objects which meets criteria defined in the 

search instructions, and wherein the database searcher is 
operable to search data stored in the database in 
accordance with search criteria relating to any one or 
more of the first or second archive data* 

15 

18. Appaxatus according to claim 6, further cos^rising 
a database searcher for searching dat^ stored in the 
database in accordance with search instructions to 
identify each part of the archive data for the three- 

20 dimensional objects which meets criteria defined in tihe 

search instructions, and wherein the database searcher is 
operable to search data stored in the database in 
accordance with search criteria relating to any one or 
more of the first or third archive data. 
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19. Apparatus according to claim 11 further comprising 
a database searcher for searching data stored in the 
database in accordance with search Instructions to 
identify each part of the archive for three-dimensional 
objects which meets criteria defined in the search 
instructions, and wherein the database searcher is 
operable to search data stored in the database in 
accordance with search criteria relating to any one or 
more of the first or fourth azrehlve data. 

20. Apparatus according to claim 1, further comprising 
a renderer for rendering a model of the three-dimensional 
objects to generate image data showing the movements of 
the objects. 



21. Apparatus for generating data for archiving data 
defining models of three-dimensional objects, comprising* 

a receiver for receiving data defining movements of 
the three-dimensional objects; and 

a first archive data generator for generating first 
archive data defining where at least one object is 
looking. 

22. Apparatus for searching data stared in a database 
comprising data for three-dimensional objects and first 
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archive dlat.a defining where the objects are looking, the 
apparatus comprising a receiver for receiving data 
defining search criteria relating to the first archive 
data, and a database searcher for searching the data in 
the database in accordance with the search criteria* 

23. Apparatus for archiving virtual meeting data, 
comprislngt 

a store for storing data defining a three- 
dimensional model of participants In the meeting; 

a receiver for receiving data defining movements of 
the participants; 

an archive data generator for generating archive 
data defining where each person Is looking; and 

a database for storing the archive data linked to 
data defining the movements of the participants* 

24* Apparatus for archiving virtual meeting data, 
comprising ft 

a store for storing data defining a three- 
dimensional model of the participants in the meeting; 

a receiver for receiving data defining movements of 
the participants; 

an archive data generator for generating archive 
data defining where each person is looking; and 
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a database for storing the archive data linked to 
data defining the three-di»ens±o„al model amended in 
accordance with the data defining the nK.veinenta. 

25. A method of archiving data defining models of three- 
dimensional objects, comprising: 

receiving data defining movements of the three- 
dimensional ob jecbs ; 

generating first archive data defining where at 
least one object is looking; and 

storing the first archive data in a database such 
that It is associated with archive data for the three- 
dimensional objects. 

26. A method of generating data for archiving data 
defining models of three-dimensional objects, comprising. 

receiving data defining movements of the three- 
dimensional objects; and 

generating first archive data defining where at 
least one object is looking. 

27. A method of searching data stored in a database 
comprising data for three-dimensional objects and first 
archive data defining where the objects are looking, 
comprising receiving data defining search criteria 
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relating to the first archive data and searching the data 
in the database in accoardance with the search criteria. 

28. A method of archiving virtual meeting data, 
comprising t 

storing data defining a three-dimensional model of 
participants in the meeting; 

receiving data defining movements of the 
participants; 

generating archive data defining where each person 
is looking; and 

storing the archive data in a database linked to 
data defining the movements of the participants. 

29. A method of archiving virtual meeting data^ 
comprising: 

storing data defining a three-dimensional model of 
the pazrticipants in the meeting; 

receiving data defining movements of the 
participants; 

generating archive data defining where each person 
is looking; and 

storing the archive data in a database linked to 
data defining the three-dimensional model amended in 
accordance with the data defining the movements. 
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30, A Storage device storing coniputier-useable 
instructions for causing a programmable processing 
apparatus to become configured as an apparatus as set out 
in at least one of claims 1, 21^ 22, 23 and 24. 

5 

31, A storage device storing computer-^useablo 
instructions for causing a programmable processing 
apparatus to become operable to perform a method as set 
out In at least one of claims 25, 26, 27, 28 and 29- 

10 

32 • A signal conveying computer-use able* Instructions for 
causing a programmable processing apparatus to becoxoe 
configured as an apparatus as set out in at least one of 
claims 1, 21, 22, 23 and 24 « 

15 

33* A Signal conveying computer-useable Instructions for 
causing a programmable processing apparatus to become 
operable to perform a method as set out in at least one 
of claims 25, 26, 27, 28 and 29. 

20 

34. Apparatus for archiving data defining models of 
three-dimensional objects, comprising s 

receiving means for receiving data defining 
movements of the three->dlmensional objects; 
25 means for generating first archive data defining 
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where at least: one object Is looking; and 

a dat:ai>ase for sHorlng -the first archive data such 
that it is associated with archive data for the three- 
dimensional objects. 

35. Apparatus for generating data for archiving data 
defining models of three-dimensional objects, comprising: 

receiving means for receiving data defining 
movements of the three-dimensional objects; and 

means for generating first archive data defining 
where at least one object is looking. 

36. Apparatus for seax^ching data stored in a database 
con^rising data for three-dimensional objects and first 
archive data defining where the objects are looking/ the 
apparatus comprising means for receiving data defining 
search criteria relating to the first archive data, and 
means for searching the data in the database in 
accordance with the search criteria- 

37. Apparatus for archiving virtual meeting data, 
comprising s 

storage means for storing data defining a three- 
dimensional model of participants in the meeting; 

means for receiving data defining movements of the 
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parti.clpant;s ; 

means for generalilng archive da-ta defining where 
each person is looking; and 

database xoeans for storing the archive da-ta linked 
to data defining the iftovetnents of the participants. 

38. Apparatus for archiving virtual meeting data^ 
comprising s 

storage means for storing data defining a three- 
dimensional model of the participants In the meeting; 

means for receiving data defining movements of the 
participants ^ 

means for generating archive data defining where 
each person is looKlng; emd 

database means for storing the archive data linked 
to data defining the three-dimensional model amended in 
accordance with the data defining the movements. 

39. Apparatus for creating a two-dimensional (2D) moving 
image database, comprising: 

an image data receiver for receiving 2D moving image 

data; 

a participant identifier for Identifying 
participants shown in the moving image data; 

a gaze direction data provider for providing data 
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relat.lng t;o the direction of gaze of participants in the 
moving image data; 

a memory for storing the database; and 
a database generator for generating in the memory a 
database^ comprising: an image data file having a 
plurality of image data storage areas each s-toring a 
respective different section of the moving image data, a 
participants data file storing data representing the 
identified participants , and a viewing data file having 
a plurality of viewing data storage areas each associated 
with a respective different one of thei image data storage 
areas and each storing gaze data for the section of 
moving image data stored in the corresponding image data 
storage area. 

40. Apparatus according to claim 39^ wherein the 
database generator is axxanged to generate the database 
BO as to provide a plurality of viewing data files each 
associated with a particular different one of 
participants shown in the moving image data and each 
having a plurality of viewing data storage areas each 
associated with a respective different image data storage 
area and each storing gaze data for the associated 
participant for the section of moving image data stored 
in the corresponding image data storage area. 



192 263S6S0 (CFF 1194 US) 

41. Apparatus accox-dlng tio claim 39 r fuirther comprising 
an audio datia receiver for receiving audio data 
corresponding to 2D moving image data, wherein the 
database generator is arranged to generate the database 
5 such tha^ the database also comprises an audio file 

having a plurality of audio data storage areas each 
associated with a respective different one of the image 
data storage areas with each audio data storage area 
storing data x-elatdLng to audio data associated with the 
10 section of the moving image data stored in the 

corresponding image data storage area. 

42* Apparatus according to claim 39 ^ further comprising 
an audio data receiver for receiving audio data 

15 corresponding to the 2D moving image data^ wherein the 

database generator is operable to generate the database 
such that the database also coinprises a plurality of 
audio files each associated with a respective different 
participant and each having a plurality of audio data 

20 storage areas each associated with a respective different 

one of the image data storage areas and each audio da-ta 
storage area storing data relating to audio data 
associated with the corresponding image data storage 
etrea. 



25 
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43. Apparatus according to claim 39/ further comprising 
a speech data receiver for receiving speech data 
corresponding to the 2P moving image data^ the database 
generator being operable to generate the database such 
that the database also comprises a speech file having a 
plurality of speech data storage areas each associated 
with a respective different one of the image data storage 
areas with each speech data storage area being arranged 
to store data relating to words spoJcen by or associated 
with a participant in image data stored in the 
corresponding image data storage area. 

44 • Apparatus according to claim 39, further comprising 
a speech data receiver for receiving speech data 
corresponding to the 2D moving image data, the database 
generator being operable to generate the database such 
that the database also comprises a plurality of speech 
files each associated with a different participant and 
each speech file having plurality of speech data storage 
areas each associated with a respective different one of 
the image data storage areas and storing data relating to 
words spoken by or associa-ted with that participant in 
relation to image data stored In that image data storage 
area. 
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45. Appara-tus according -to claLim 39, further comprising 
a speech data receiver for receiving speech data 
corresponding to the 2D moving image data, the database 
generator being operable to generate the database such 
that the database also comprises a speech file having a 
plurality of speech data storage areas each associated 
with a respective different one of the image data storage 
areas with each speech data storage area storing data 
relating to words spoken by or associated with a 
paxrticipant in dUnage data stored in the corresponding 
image data storage area, the database generator also 
being operable to generate the viewing data file such 
that each viewing data storage area stores data 
indicating which, if any, of other participants in an 
item of image data stored in the corresponding Ijnage data 
storage area is being gazed or looked at by the 
participant when the participant associated with the 
viewing data file is speaking* 

46, Apparatus according to claim 45, wherein the 
database generator is operable to generate a viewing data 
file for each of a plurality of participants shown in the 
moving image data such that each viewing data storage 
area of each viewing data file stores data indicating 
which, if any, of other participants in an item of linage 
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da^a S'tored In tJie corresponding ImagQ dat.a stiorage area 
is being looked at: by the part.lclpanb associated with 
thai: viewing data file when that participant Is speaking. 

5 47. Apparatus according to claim 43^ wherein the speeph 

data receiver further comprises a converter for 
converting the audio speech data to text. 

48. Apparatus according to claim 47 ^ wherein the 
10 converter comprises a speech recognition data file, a 

speech processor for using the speech recognition data 
file to identify phonemes in the received audio speech 
data, a recognition processor for recognising words in 
the speech data and a writer for storing both the 
15 Identified phonemes and text representing the identified 

words In the speech data files* 

49* J^paratus according to claim 43 , wherein the speech 
data receiver further comprises a transcript retriever 

20 for accessing a transcript of speeches occurring in the 

moving Image data^ eui Identifier for Identifying speeches 
by different participants in the transcript and a writer 
for storing each Identified speech in the speech file of 
the participant responsible for that speech such that 

25 each speech data storage area of that speech contains the 
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portion of the transcript: associated with the image data 
stored in the corresponding Ijnage data storage area. 

50. Apparatus according to claim 43, further comprising 
a histogram generator for using the speech and gaze data 
to generate for each speech a viewing histogram 
indicating the amount of tiaae the speaker looked at each 
of the other participants during that speech, the 
database generator being operable to generate in the 
database a respective viewing histogram file for each 
participant and to store the generated viewing histograms 
in the viewing histogram file such that each viewing 
histogram la associated with the speech data storage 
areas where the corresponding speech is stored. 

51. Apparatus according to claim 39, wherein the 
participant Identifier comprises a display generator for 
displaying a section of the moving image data to a user, 
an input device for enabling input to the dat:abase 
generator by the user of data identifying the 
participants and an input data reader for reading the 
data input by the user. 



52. Apparatus according to claim 39, wherein the gare 
direction data provider comprises a display generator for 
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displaying the moving image data to a user, an input 
device for enabling the user to enter to the database 
generator the direction of gaze of the participants in 
the displayed image data and an input data reader for 
reading the data entered by the user. 

53. Apparatus according to claim 39, wherein the Unage 
data receiver is arranged to receive video data. 

54. Apparatus according to claim S3, further comprising 
a data compressor for causing the 2D moving image data to 
be stored in a compressed format in the image data file. 

55. Apparatus according to claim 54, wherein the data 
compressor is arranged to store the 2D moving image data 
in MP£G format. 



56. A method of creating a two-dimensional (2D) moving 
image database, comprisiagj 

receiving 2D moving image data; 

identifying participants shown in the moving Image 

data; 

providing data relating to the direction of gaze of 
participants in the moving image data; and 

using a processor to generate in a memory a 
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datiabase, comprising: an image data file having a 
plurality of image data storage areas each storing a 
respectiive different section of the moving image data, a 
participants data file storing the identified 
parrticipants^ ajid a viewing dalia file having a plurality 
of viewing datia storage areas each associated with a 
respective different one of the image data storage areas 
and each storing gaze data for the section of moving 
image data stored in the corresponding Image data storage 
area* 

57. i^paratus for providing two-dimensional (2D) moving 
image data for storage in a database having a database 
structure comprising an image data file having a 
plurality of image data storage areas each arranged to 
store a x-espective dif f erent section of the moving image 
data^ a participants data file arranged to store 
participants data, and a viewing data file having a 
plurality of viewing data storage areas each associated 
with a respective different one of the image data storage 
areas and each arranged to store gaze data for the 
section of moving image data stored in the corresponding 
image data storage area, the apparatus comprising: 

an image data receiver for receiving 2D moving image 

data; 
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a participant identifier for identifying 
participants shown in the moving image data; and 

a gaze direction data provider for providing data 
relating to the direction of ga^e of participants in the 
moving image data,. 



58. Apparatus for providing two-din«n»ional (2D) moving 
io^ge data for storage in a database having a database 
structure comprising an image data file having a 
plurality of l«age data storage areas each arranged to 
store a respective different section of the moving ijoage 
data, a participants data file arranged to store 
participants data, and a plurality of viewing data files 
each associated with a particular different one of 
participants shown in the xnovlng l«age data and each 
having a plurality of viewing data storage areas each 
associated with a respective different isage data storage 
area and each arranged to store gaze data for the 
associated participant for the section of moving image 
data stored in the corresponding image data storage area, 
the apparatus coa^risingt 

an image data receiver for receiving 2D moving image 

data; 

a participant identifier for identifying 
participants shown in the moving image data; and 
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a gaze direction data provider for providing data 
relating to the direction of gaze of participants in the 
moving image data» 

5 59 • Apparatus according to claim 58 for providing 2D 

moving image data for storage xn a database structure 
which also comprises an audio file having a plurality of 
audio data storage areas each associated with a 
respective different one of the image data storage areas 

10 with each audio data storage area arranged to store data 

relating to audio data aesociated with the section of the 
moving image data stored in the corresponding Image data 
storage area, the apparatus further comprising an audio 
data receiver for receiving audio data corresponding to 

15 2D moving Image data. 

60. Apparatus according to claim 58, for providing 2d 
moving image data for storage in a database structure 
which also comprises a plurality of audio files each 

20 associated with a respective different participant and 

each having a plurality of audio data storage areas each 
associated with a respective different one of the image 
data storage areas and each audio data storage area being 
arranged to store data relating to audio data associated 

25 with the corresponding image data storage - area, the 
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apparatus further aomprLsLng an audio data receiver for 
receiving audio data corresponding to the 2D moving image 
data. 

61. Apparatus according to claim 58, for providing 2D 
moving image data for storage in a database structure 
which also comprises a speech file having a plurality of 
speech data storage areas each associated with a 
respective different one of the image data storage areas 
with each speech data storage area being arranged to 
store data relating to words spoken by or associated with 
a participant in image data stored in the corresponding 
image data storage area, the apparatus further comprising 
a speech data receiver for receiving speech data 
corresponding to the 20 moving image data* 

62. Apparatus according to cladLm 58^ for providing 2D 
moving image data for storage in a dat£d3ase structure 
which also comprises a plurality of speech files each 

20 associated with a different paxrticipant and each speech 

file having plurality of speech data storage areas each 
associated with a respective different one of the image 
data storage areas amd each arranged to store data 
relating to words spoken by or associated with that 

25 participant in relation to image data stored in that 
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image data storage area, the apparatus comprising a 
speech data receiver for receiving speech data 
corresponding to the 2D moving image data, 

63. Apparatus according to claim 57, for providing 2D 
moving image data for storage in a database structure 
which also comprises a speech file having a plurality of 
speech data storage areas each associated with a 
respective different one of the image data storage areas 
with each speech data storage area being arranged to 
store data relating to words spoken by or associated with 
a participant in image data stored in the corresponding 
image data storage area, and in which each viewing data 
storage area is arranged to store data indicating which, 
if any, of other participants in an item of image data 
stored in the corresponding image data storage area is 
being gazed or looked at by the participant when the 
participant associated with the viewing data file is 
speaking, the apparatus further comprising a speech data 
receiver for receiving speech data corresponding to the 
2D moving image data. 



64. Apparatus according to claim 63, for providing 2D 
moving image data for storage in a database structure 
which includes a viewing data file for each of a 
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plurality of participants shown in the moving linage data 
such that each viewing data storage area of each viewing 
data file Is arranged to store data Indicating which ^ if 
any, of other participants In an item of Image data 
5 stored In the corresponding image data storage area is 

being looked at by the participant associated with that 
viewing data file when that participant is speaking. 

65* Appara-tus according to claims 61, wherein the speech 
10 data receiver comprises a converter for converting audio 

speech data to text* 

66- J^paratus according to clelm 65^ wherein the 
converter comprises a speech recognition data file, a 

15 speech processor for using the speech recognition data 

file to identify phonemes in the received audio speech 
data, a word recogniser for recognising words in the 
speech data and a writer for causing both the identified 
phonemes and text representing the Identified words to be 

20 stored In the speech data files. 

67 » Apparatus according to claim 61, wherein the speech 
data receiver comprises a transcript retriever for 
accessing a transcript of speeches occurring In the 
25 moving image data, an Identifier for Identifying speeches 
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by dlff ereni: pajri:lci.pant;8 in th« liranscrlp^ and a writer 
for causing each identified speech to be stored in the 
speech file of the participant responsible for that 
speech such that each speech data storage area of that 
speech contains the portion of the transcript associated 
with the image data stored in the corresponding image 
data storage area* 



68. Apparatus according to claim 61, further con^rising 
a histograjn generator for using the speech and gaze data 
to generate for each speech a viewing histogram 
indicating the amount of time the speaker looXed at each 
of the other participants during that speech for storage 
in a respective viewing histogram file in the database 
such that each viewing histogram is associated with the 
speech data storage areas where the corresponding speech 
is stored. 



69 • Apparatus according to claim 57, wherein the 
participant identifier comprises a display generator for 
displaying a section of the moving Image data to a user, 
an input device for enabling input to the database 
generator by the user of data identifying the 
participants and an input data reader for reading the 
data input by the user. 
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70. Apparatius according to claim 57, wherein 'the gaze 
direction dat.a provider comprises a display generator for 
displaying the moving image datia to a user, an input 
device for enabling the user to enter to the datal^ase 
5 generator the direction of gaze of the participants in 

the displayed image data and an input dat.a reader for 
reading the data entered by the user. 

71 » Apparatus according to claim 57, wherein the image 
10 data receiver is arranged to receive video data. 

72. Apparatus according to claim 71, further comprising 
a data compressor for causing the 2D moving image data to 
be stored in a compressed format in the image data file* 

15 

73. Apparatus according to claim 72, wherein the data 
compressor is arranged to store the 2D moving image data 
in MPEG format* 

20 74. A method of providing a two-dimensional (2D) moving 

image for storage in a database having a database 
structure comprising an image data file having a 
plurality of image data storage areas each sirranged to 
store a respective different section of the moving image 

25 data^ a participants data file arranged to store 
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participants data, and a viewing data file having a 
plurality of viewing data storage areas each associated 
with a respective different one of the image data storage 
areas and each arranged to store gaze data for the 
section of moving ijnage data stored in the corresponding 
image data storage area, the method comprising: 
receiving 2D moving image data? 

identifying participants shown in the moving image 
data; and 

providing data relating to the direction of gaae of 
participants in the moving image data. 

75. A method of providing a two-dlmenaionaX (2D) moving 
image for storage in a database having a database 
structure comprising an image data file having a 
plurality of image data storage areas each arranged to 
store a respective different section of the moving image 
data, a participants data file arranged to store 
participants data, and a plurality of viewing data files 
each associated with a particular different one of 
participants shown in the moving image data and each 
having a plurality of viewing data storage areas each 
associated with a respective different image data storage 
area and each arranged to store gaze data for the 
associated participant for the section of moving image 
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data stored Ixi the corresponding Image data storage area, 
the method comprising: 

receiving 2D moving image data; 

identifying participants shown in the moving image 
data ; and 

providing data relating to the direction of gaze of 
participants in the inoving image data. 

76. Apparatus for searching a dated>ase generated by 
using apparatus in accordance with claim 39 , comprising? 

a search parameter receiver for receiving a first 
search parameter identifying a first participant and a 
second search parameter identifying a second participant 
at which the first participant is looking; 

a data file identifier for Identifying in the 
database a viewing data file associated with the first 
participant ? 

a viewing data storage area identifier for 
identifying within the identified viewing data file any 
viewing data storage area storing data associating the 
second participant with the first participant; 

an image data storage area identifier for 
identifying an image data storage area associated with an 
identified viewing data storage area; and 

a data supplier for supplying to a user details 
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identifying the identified image data storage area. 

11. i^paratus according to claim 76, for searching a 
database r wherein the search parameter receiver is 
arranged for receiving a further search parameter 
defining text spoken by the first participant to the 
second participant, wherein the apparatus further 
comprises a speech data storage area identifier meems for 
identifying within the speech file associated with the 
first participant any speech data storage az^ea containing 
that speech, and wherein the image data storage area 
identifier is further arranged for identifying an Image 
data storage area corresponding to an identified speech 
data storage area* 

78 » Apparatus according to claim 77, which further 
comprises a viewing proportion file checker for checking 
the viewing proportion file for the first participant for 
the identified speech and for disregarding any identified 
speeches where the amount of time the first participant 
looks at the second participant is less than a 
predetermined proportion of the duration of that speech. 

79. A method of searching a database In accordance with 
claim 56, comprising receiving a first search parameter 
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Identifying a first participant and a second search 
parameter identifying a second participant at which the 
first participant is looking; identifying in the database 
a viewing data file associated with the first 
participant^ identifying within the identified viewing 
data file any viewing data storage area storing data 
associating the second participant with the first 
participant? identifying an image data storage area 
associated with an Identified viewing data storage area? 
and supplying to a user details identifying the 
identified Image data storage area. 

80. A signal carrying proceeeor in^lementable 
instructions for controlling a processor to carry out the 
steps as clairoed in the method of at least one of claims 
56^ 74^ 75 and 79. 

81. A Storage medium storing processor implementahle 
instructions for controlling a processor to carry out the 
steps as claimed in the method of at least one of claims 
56, 74, 75 and 79- 

82. Apparatus for creating a two-dimensional (20) moving 
image database, comprising: 

means for receiving 2D moving image data? 
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means for identLifying participants shora in the 
moving image data; 

means for providing data relating to the direction 
of gaze of participants in the moving image data; 

memory means for storing the database; and 

processing means for generating in the memory a 
database^ comprising? an image data file having a 
plurality of image data storage areas each storing a 
respective different section of the moving image data, a 
participants data file storing data representing the 
identified participants, and a viewing data file having 
a plurality of viewing data storage areas each associated 
with a respective different one of the image data storage 
areas and each storing gaze data for the section of 
moving image data stored in the corresponding Image data 
storage area. 

83. Apparatus for providing two-dimensional (20) moving 
image data for storage in a database having a database 
structure comprising an image data file having a 
plurality of image data storage areas each arrwged to 
store a respective different section of the moving image 
data, a participants data file arranged to store 
participants data, and a viewing data file having a 
plurality of viewing data storage areas each associated 



211 2S35650 (CFP 1194 US) 



with a respectilve dlfferdni: one of the Image data storage 
areas and each arranged to store gaze data for the 
section of moving Image data stored in the corresponding 
image data storage area, the apparatus comprising: 

means for receiving 2D moving image data; 

means for identifying participants shown in the 
moving image data; and 

means for providing data relating to the direction 
of gaze of participants in the moving image data. 

84 ♦ Apparatus for providing two-dimensional (2D) moving 
image data for storage in a database having a database 
structure comprising an image data file having a 
plurality of image data storage areas each arranged to 
store a respective different section of the moving image 
data, a participants data file arranged to store 
participants data, and a plurality of viewing data files 
each associated with a particular different one of 
participants shown in the moving image data and each 
having a plurality of viewing data storage areas each 
associated with a respective different image data storage 
area and each arranged to store gaze data for the 
associated participant for the section of moving image 
data stored in the corresponding Image data storage area, 
the apparatus comprising s 
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means for receiving 2D moving Image dat:a; 

means for Identilfylng partlcipajn-bs shown In the 
moving Image dana; and 

means for providing datia relatilng to the direction 
of gaze of partlclpsmts in the moving image data. . 

85. Apparatus for searching a database generated by 
using apparatus In accordance with claim 39, comprising: 

means for receiving a first search parameter 
identifying a first participant and a second search 
parameter identifying a second participant at which the 
first participant is looking; 

means for Identifying in the dateUdase a viewing data 
file associated with the first participant; 

means for Identifying within the Identified viewing 
data file any viewing data storage area storing data 
associating the second participant with the first 
participant ; 

means for Identifying an image data storage area 
associated wltti an identified viewing data storage area; 
and 

means for supplying to a user details Identifying 
the Identified image data storage area* 

• 96. Apparatus for archiving still image data. 
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comprising! 

an ijnage da^a receiver for receiving Image dalia 
defining an Input, image; 

a first archive data generator for generating first, 
archive data defining a person or animal in the input 
image; 

a second archive data generator for generating 
second archive data defining a person, animal or object 
at which the person or animal defined In the first 
archive data Is looking; and 

a database for storing the first and second archive 
data such that it Is associated with Image data defining 
the input Image. 

87. Apparatus according to claim 86, further comprising 
a third archive data generator for generating third 
archive data defining text, and wherein the database is 
arranged to store the third archive data such that It Is 
associated with the first and second archive data and the 
stored image data defining the input JLmage. 

88. Apparatus according to claim 87, wherein the third 
archive data generator comprises an optical character 
recognlser for performing optical character recognition. 
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89. Apparat^us according t.o claSja 87, wherein the third 
archive data generator is operable to allow a user to 
input the text information manually. 

90. Apparatus according to claim 86, further comprising 
a fourth archive data generator for generating fourth 
archive data defining information about when the input 
image was recorded, and wherein the database is arranged 
to store the fourth archive data such that it is 
associated with the stored image data defining the input 
image and related archive data. 

91. Apparatus according to claim 90, wherein the fourth 
archive data generator comprises a recording date reader 
for performing processing to read a recording date from 
the received image data« 

92. Apparatus according to claim 90, wherein the fourth 
archive data generator is operaJdle to allow a user to 
input manually the information about when the input image 
was recorded. 
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93. Apparatus according to claim 86, wherein the first 
archive data generator comprises an image recognition 
processor for perforoiing image recognition processing. 
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94. Apparatus acoordlng claim 86, wherein t:he first 
archive data genera^tor Is operable to allow a user to 
Input the first archive data zoanually. 

95. Apparatus according to claloi 86, whex-eln the second 
archive data generator is operable to allow a user to 
Input the second archive data manually. 

96. Apparatus according to claim 86^ further comprising 
a database searcher for searching data stored in the 
database in accordance with search instructions to 
identify each stored image which meets criteria defined 
in the seairch Instructions. 

97. Apparatus according to claim 96, wherein the 
database searcher is operable to search data stored in 
the database In accordance with search criteria relating 
to the second archive data. 

98* Apparatus according to claim 97, wherein the 
database searcher is operable to sesurch data stored in 
the database in accordance with search criteria relating 
to the first and second archive data. 

99. Apparatus according to claim 96, wherein the 
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da-tabase searcher is operable tio search dat:a stored in 
the database in accordance with search criteria relating 
to any one or more of the first, second, third or fourth 
archive data. 

100. Apparatus according to claim 86, further comprising 
a display for displaying an image to a user» 

101 • Apparatus for generating data for archiving still 
image data in a database, comprising: 

an image data receiver for receiving image data 
defining an input image; 

a first archive data generator for generating first 
archive data defining a person or animal in the input 
image; and 

a second archive data generator for generating 
second archive data defining a person, animal or object 
at which the person or animal defined in the first 
archive data is looking. 

102. Apparatus for searching data stored in a database 
comprising image data defining a plurality of still 
dLmages and, for each of the still images, first archive 
data defining a person or animal in the input image and 
second archive data defining a person, andLmal or object 
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at whic=h the person or animal defined In the first 
archive data is looking, the apparatus comprising a 
receiver for receiving data defining search criteria, and 
a searcher for searching the data in the database in 
accordance with search criteria relating to the second 
archive data to Identify each stored image which meets 
the criteria. 

103. A method of archiving still Image data in a computer 
database, comprising the steps of s 

receiving image data defining an input image; 

generating first archive data defining a person or 
animal in the input image; 

generating second archive data defining a person^ 
animal or object at which the person or animal defined in 
the first archive data is looking? and 

storing the first and second archive data in the 
database such that it is associated with Ijnage data 
defining the input image* 

104. A method of generating data for archiving still 
image data in a database r comprising! 

receiving image data defining an input image; 

generating first archive data defining a person or 
animal in the input image; and 



218 2635650 (CFP 1194 US) 



generating second archive datia defining a person, 
animal or object at which the person or animal defined in 
the first archive data Is looking. 

105, A method of searching data stored In a database 
comprising Image data defining a plurality of still 
Images and, for each of the still Imagea, first archive 
data defining a person or animal in the input Image and 
second archive data defining a person^ animal or object 
at which the person or aniinal defined in the first 
archive data is looking/ the method comprising receiving 
data defining search criteria and searching the data in 
the database in accordance with search criteria relating 
to the second archive data to Identify each stored Ixaage 
which meets the criteria. 

106. A storage device storing computer-useable 
instructions for causing a programmable processing 
apparatus to become configured as an apparatus as set out 
in at least one of claims 86 r 101 and 102. 

107- A storage device storing con^uter-useabXo 
Instructions for causing a programmable processing 
apparatus to become operable to perform a method as set 
out in at least one of claims 103^ 104 and 105. 



219 



2635650 (CFP 1194 US) 



108. A signal conveying compuHer-^useable insHructioas for 
causing a prograxnmabXe processing apparatus "to become 
configured as an apparatus as set out in at least one of 
claims 86, 101 and 102. 

5 

109. A signal conveying computer-useable instructions for 
causing a programmable processing apparatus to become 
operable to perform a method as set out in at least one 
of claims 103^ 104 and 105 • 

10 

110* Apparatus for archiving still image data^ 
comprising: 

means for receiving image data defining an input 
image ; 

IS means for generating first archive data defining a 

person or animal in the input image; 

means for generating second archive data defining a 

person r animal or object at which the parson or animal 

defined in the first archive data is looking; and 
20 a database for storing the first and second archive 

data such t.hat it is associated with image data defining 

the input image » 
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111. Apparatus for generating data for archiving still 
Image data in a database, ccmprlslng: 
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means for receiving Image data defining an inpu^ 
Image; 

means for generating first: archive data defining a 
person or animal in the Input image; and 

means for generating second archive data defining a 
person^ animal or object at which the person or animal 
defined in the first archive data is looking * 

112 • Apparatus for searching data stored in a database 
comprising image data defining a plurality of still 
Images and/ for each of the still images, first archive 
data defining a person or animal in the input image and 
second archive data defining a person r animal or object 
at which the person or animal defined in the first 
archive data is looking , the apparatus comprising means 
for receiving data defining search criteria and means for 
searching the data In the database In accordance with 
search criteria relating to the second archive data to 
identify each stored image which meets the criteria. 

113. Apparatus for generating a database structure, 
comprising a memory for storing data and a processor 
operable to generate In the memory a database structure 
ccHnprislngt an image data file having a plurality of 
image data storage areas each arranged to store a 
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respective differ-ent l-bem of image data; a paxtlclpants 
data file arranged to store data identifying participants 
shown in image data stored in the image data file; and a 
viewing data file having a plurality of viewing data 
5 storage areas each associated with a respective different 

one of the image data storage areas and being arranged to 
store data relating to the direction In which a 
pasrticipant shown in an item of image data stored in the 
corresponding image data storage area is looking.. 

10 

114* Apparatus according to claim 113, wherein the 
processor is operable to generate in the database 
structure a plurality of viewing data files each arranged 
to be associated with a particular different one of 

15 participants shown in the image data and each having a 

plurality of viewing data storage areas each associated 
with a respective different Image data storage area and 
each being arranged to store data indicating which, if 
any, of the other participants the participant associated 

20 with that viewing data storage area is looking at in an 

Item of Image data stored in the corresponding image data 
storage area. 
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115<. Apparatus according to claim 113, wherein the 
participants are people » 
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116. Apparaiius according to claim 113, wherein the 
processor la operable to generate the database structure 
such that the database structure also comprises an audio 
file having a plurality of audio data storage areas each 
associated with a respective different one of the linage 
data storage areas with each audio data storage area 
being arranged to store data relating to audio data 
associated with the corresponding image data storage 
area. 

117* Apparatus according to claim 113, wherein the 
processor is operable to generate the dateibase structure 
such that the database structure also con^rises a 
plurality of audio files each arranged to be associated 
with a respective different paxrticipant and each having 
a plurality of audio data storage areas each associated 
with a respective different one of the image data storage 
areas and each audio data storage area being arranged to 
store data relating to audio data associated with the 
corresponding image data storage area. 

118. Apparatus according to claim 113, wherein the 
processor is operable to generate the database structure 
such that the database structure also comprises a speech 
file having a plurality of speech data storage areas each 
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associa-ted with a respective different one of the image 
data storage areas with each speech data atorage area 
being arranged to store data relating to words spoken by 
or associated with a participant in an item of image data 
stored in the corresponding image data storage area. 

119. Apparatus according to claim 113, wherein the 
processor is operable to generate the database structure 
such that the database structure also comprises a 
plurality of speech files each associated with a 
particular different participant and each speech file 
having plurality of speech data storage areas each 
associated with a respective different one of the image 
data storage areas and being arranged to store data 
relating to words spoken by or associated with that 
participant in relation to an image stored in that image 
data storage area* 

120. Apparatus according to claim 113 r wherein the 
processor is operable to generate the database structure 
such that the database structure also comprises a speech 
file having a plurality of speech data storage areas each 
associated with a respective different one of the image 
data storage areas with each speech data storage area 
being arranged to store data relating to words spoken by 
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or associated w±th a paz^iclpant- In an l^em of Image data 
stored In the corresponding Image data storage area and 
to generate the viewing data file such that each viewing 
data storage area Is arranged to store data Indicating 
which, if any, of other participants In an item of image 
data stored In the corresponding Ixaage data storage area 
Is being looked at by the participant when the 
participant associated with the viewing data file Is 
speaking . 

121, Apparatus according to claijn 120, wherein the 
processor is operable to generate a viewing data file for 
each of a plurality of participants shown in the image 
data with each viewing data storage area of each viewing 
data file being arranged to store data indicating which, 
if any, of other participants in an item of image data 
stored in the corresponding image data storage area is 
being looked at by the participant associated with that 
viewing data file when that participant is speaking. 

122. ^paratus according to claim 118, wherein the 
processor is operable to generate the database structure 
such that the database structure also ccmiprises at least 
one viewing proportion file arranged to store data 
relating to the amount of time an associated participant 
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looks at each of a plurality of other participants while 
that participant is speaking. 

123. Apparatus according to claim 118^ wherein the 
processor is operable to generate the database structure 
such that the speech Is arranged to be stored as text. 

124. Apparatus according to claim 113, wherein the 
processor is operable to generate the database structure 
such that each image data storage area is arranged to 
store at least one fraune of image data* 

125 « Apparatus according to claim 113^ wherein the image 
data file is arranged to store image data having sound 
data associated therewith* 

126. Apparatus for generating a database, comprising a 
memory and a processor operable to generate in the memory 
a database comprising: an image data file having a 
plurality of Image data storage areas each storing a 
respective different item of image data; a participants 
data file storing data identifying participants shown in 
the image data stored in the image data file; and a 
viewing data file having a plurality of viewing data 
storage areas each associated with a respective different 
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one of the Image data storage areas and storing data 
identifying the direction in which a participant shown in 
the item of image data stored in the corresponding Image 
data storage area is looking • 

127. A database structure coznprlslngs an image data file 
having a pluraaity of linage data storage areas each 
arranged to store a respective different item of linage 
data; a participants data file arranged to store data 
identifying participants shown in image data stored in 
the image data file; and a viewing data file having a 
plurality o£ viewing data storage areas each associated 
with a respective different one of the image data storage 
areas and being arranged to store data identifying the 
direction in which a participant shown In the image 
represented by an item of image data stored in the 
corresponding image data storage area Is looking. 

12 8. A database comprising: an image data file having a 
plurality of image data storage areas each storing a 
respective different item of image data; a participants 
data file storing data identifying pairticlpants shown in 
the linage data stored in the image data file; and a 
viewing data file having a plurality of viewing data 
storage areas each associated with a respective different 
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one of the image data sHorage areas and storing data 
identifying the direction in which a participant shown in 
the iiaage represented by the item of image data stored in 
the corresponding image data storage area is looking. 

129. A signal carrying a database structure in accordance 
with claim 127 or a database in accordance with claim 
128. 



130 » A data storage medium storing a database structure 
in accordance with claim 127 or a database in accordance 
with claim 128. 



131. A method of generating a database, comprising the 
steps of causing a processor to generate in a memory a 
database comprising; an image data file having a 
plurality of image data storage areas each storing a 
respective different item of image data; a participants 
data file storing data identifying participants shown in 
the image data stored in the image data file; and a 
viewing data file having a pluzrality of viewing data 
storage areas each associated with a respective different 
one of the image data storage areas and storing data 
identifying the direction in which a participant shown in 
the image represented by the item of image data stored in 
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the correspondxng Image data storage area is looking. 

132. Apparatus for searching a database in accordance 
with claim 128^ comprising a receiver for receiving a 
first search parameter identifying a first participant 
and a second search parameter identifying another person 
at whcan the first participant is looking; a viewing data 
file identifier for identifying in the database a viewing 
data file associated with the first participant; a 
viewing data storage area Identifier for identifying 
within the identified viewing data file any viewing data 
storage area storing data associating the second 
participant with the first participant; an image data 
storage area identifier for identifying an image data 
storage area associated with an identified viewing data 
storage area; and a data supplier for supplying to a user 
details identifying the identified image data storage 
area. 

133 • A signal carrying processor is^Xementable 

instructions for controlling a processor to carry out the 
steps as claimed in the method of claim 131. 
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134. A storage medium storing processor implementable 
instructions for controlling a processor to carry out the 
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sneps as claijn0d in -the method of claim 131- 
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^STRACT 

IMAGE PROC^ fff^TMf; APPARATUS AND DATABASE 



Three-dimensional computer model data, moving image data 
or still image data showing at least one person is stored 
in an archive database 126, 703, 8SQ, 1303, together with 
additional information to improve the searching and 
retrieval of data therefrom. The additional information 
includes view parameter data 512, 1040 which defines at 
whom or what each person is looking during each 
predetermined period of time or image. Text data 504, 
1020 which comprises words associated with the person, 
and viewing histogram data 540 which, for each period of 
text data defines the percentage of time that the 
speaking person spent looking at each other person or 
object, may also be stored^ 
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